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Project  Summary 

Multi-frame  blind  deconvolution  (MFBD)  requires  solving  an  optimization  problem  with  an 
objective  function  that  may  have  many  local  minima.  Standard  numerical  optimization  methods 
may  become  trapped  in  one  of  these  local  minimum  points,  resulting  in  poor  image  reconstructions. 
The  aim  of  this  proposed  research  is  to  develop  new  computational  approaches  that  help  to  over¬ 
come  the  local  minimum  trap  for  iterative  MFBD  algorithms.  Success  in  this  research  requires  new 
approaches  in  several,  interrelated  areas,  including:  obtaining  a  good  initial  guess,  improved  regu¬ 
larization  methods  and  approaches  for  choosing  regularization  parameters,  mathematical  analyses 
and  efficient  computational  methods  to  reduce  the  parameter  space,  and  development  of  efficient 
implicit  filtering  algorithms  for  large  scale  optimization  problems. 

In  the  three  years  of  this  grant,  we  have  developed  software,  investigated  the  use  of  implicit 
filtering  methods  to  avoid  local  minima,  considered  Gaussian  Markov  random  fields  to  improve 
regularization,  and  we  have  developed  methods  for  sparsity  constraints  that  can  be  used  for  large 
scale  MFBD  problems.  The  work  done  on  this,  and  a  previous  AFOSR  grant,  initiated  a  very 
productive  collaboration  with  Dr.  Stuart  Jefferies,  IFA,  University  of  Hawaii,  Dr.  Douglas  Hope, 
U.S.  Air  Force  Academy,  and  Dr.  Michael  Hart,  Physics  and  Astronomy,  University  of  Arizona. 


Accomplishments  for  Period:  March  15,  2012  —  March  14,  2015 
Graduate  Student  Support.  This  grant  was  used  to  partially  support  three  graduate  students: 


Student 

^  years  in  PhD  program 

Months  of  support 

Qing  Chu 

5 

8 

Sebastian  Berisha 

5 

6.75 

Keith  Kelly 

2 

6 

Clarissa  Garvey 

2 

6 

Qing  Chu  completed  her  PhD  in  summer  2013,  and  Sebastian  Berisha  completed  his  PhD  in  summer 
2014.  Keith  Kelly,  who  worked  with  the  PI  as  an  undergraduate  and  a  first  year  Masters  student, 
is  now  a  PhD  student  at  the  University  of  Texas,  Austin.  Clarissa  is  outstanding  second  year 
graduate  student. 


In  addition,  Qing  Chu  received  Honorable  Mention  for  her  paper  Wavefront  Reconstruction  in 
Adaptive  Optics  Imaging  at  the  Copper  Mountain  Conference  on  Iterative  Methods,  March  25-29, 
2012.  Qing  also  gave  an  oral  presentation  at  the  conference. 

Keith  Kelly  presented  a  poster  at  the  2013  AMOS  Conference,  September  10-13,  2013. 

Clarissa  Garvey  and  Sebastian  Berisha  participated  in  the  Trilinos  User  Group  Meeting  at 
Sandia  National  Laboratory,  Albuquerque,  NM,  November  4-5,  2014. 

Publications. 

The  following  papers  were  written  in  the  hrst  two  years  of  this  grant: 

•  A.  Cornelio,  E.  Loli-Piccolomini,  and  J.  G.  Nagy.  Constrained  Variable  Projection  Method 
for  Blind  Deconvolution,  Journal  of  Physics:  Conference  Series,  volume  386,  issue  1,  pages 
012005,  2012. 

•  S.  Berisha  and  J.  G.  Nagy.  Iterative  Methods  for  Image  Restoration,  Academic  Press  Library 
in  Signal  Processing:  Image  Video  Processing  and  Analysis,  Hardware,  Audio,  Acoustic  and 
Speech  Processing,  Academic  Press,  editor  S.  Theodoridis,  September  2013. 

•  Q.  Chu,  S.  M.  Jefferies,  and  J.  G.  Nagy.  Iterative  Wavefront  Reconstruction  for  Astronomical 
Imaging,  SIAM  J.  Sci.  Gomput.,  volume  35,  issue  5,  pages  S84-S103,  2013. 

•  J.  M.  Bardsley,  M.  Howard,  and  J.  G.  Nagy.  Image  Deblurring,  Gaussian  Markov  Random 
Fields,  and  Neumann  Boundary  Conditions,  Elect.  Trans.  Numer.  Anal.,  volume  40,  pages 
476-488,  2013. 

•  S.  M.  Jefferies,  D.  A.  Hope,  M.  Hart  and  J.  G.  Nagy,  High-resolution  imaging  through  strong 
atmospheric  turbulence.  Proceedings  of  the  SPIE,  volume  8890,  doi:10. 1117/12. 2028193,  Oc¬ 
tober  25,  2013. 

•  K.  W.  Kelly  and  J.  G.  Nagy.  Parallel  Implementation  of  a  Frozen  Flow  Based  Wave-Front 
Reconstructor,  AMOS  Technical  Conference,  Kihei,  HI,  2013. 

•  A.  Cornelio,  E.  Loli-Piccolomini,  and  J.  G.  Nagy.  Constrained  Numerical  Optimization  Meth¬ 
ods  for  Blind  Deconvolution,  Numerical  Algorithms,  volume  65,  issue  1,  pages  23-42,  2014. 

•  P.  C.  Hansen,  J.  G.  Nagy  and  K.  Tigkos.  Rotational  Image  Deblurring  with  Sparse  Matrices, 
BIT  Numerical  Mathematics,  54,  pages  649-671,  2014. 

•  S.  Gazzola  and  J.  G.  Nagy.  Generalized  Arnoldi-Tikonov  Method  for  Sparse  Reconstruction, 
SIAM  J.  Sci.  Comput.,  volume  36,  issue  2,  pages  B225-B247,  2014. 

•  J.  G.  Nagy,  S.  M.  Jefferies,  M.  Hart,  and  D.  A.  Hope.  Fast  Tomographic  Reconstruction  of 
Atmospheric  Turbulence  from  Micro-lens  Imagery  Proceedings  of  the  Advanced  Maui  Optical 
and  Space  Surveillance  Technologies  (AMOS)  Conference,  September  9-12,  2014. 

•  S.  Berisha,  J.  G.  Nagy  and  R.  J.  Plemmons.  Estimation  of  Atmospheric  PSF  Parameters  for 
Hyperspectral  Imaging.  Numer.  Lin.  Alg.  Apph,  to  appear,  2015. 

•  S.  Berisha,  J.  G.  Nagy  and  R.  J.  Plemmons.  Deblurring  and  Sparse  Unmixing  of  Hyperspectral 
Images  using  Multiple  Point  Spread  Functions.  SIAM  J.  Sci.  Comput.,  to  appear,  2015. 
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Conference,  Colloqnia,  and  Seminar  Activities. 

The  following  talks/lectures  were  given  by  PI  Nagy: 

•  Separable  Nonlinear  Inverse  Problems  and  the  Local  Minimum  Trap.  Copper  Mountain  Con¬ 
ference  on  Iterative  Methods. 

Copper  Mountain,  CO,  March  25-29,  2012.  Contributed  Talk. 

•  Implicit  Filtering  Methods  for  Inverse  Problems.  SIAM  Conference  on  Applied  Linear  Alge¬ 
bra.  Valencia,  Spain,  June  16-23,  2012. 

Invited  Minisymposium  Talk. 

•  Large  Scale  Inverse  Problems  in  Image  Reconstruction.  SAMSI  Opening  Workshop  on  Mas¬ 
sive  Datasets  Program.  Research  Triangle  Park,  Durham,  NC,  September  9-12,  2012. 

Invited  Plenary  Talk. 

•  Wavefront  Reconstruction  in  Adaptive  Optics  Imaging.  VA  Tech  Department  of  Mathematics 
Colloquium.  Blacksburg,  VA,  November  30,  2012. 

Invited  Talk. 

•  Hybrid  Krylov  Subspace  Methods  for  Inverse  Problems.  New  Frontiers  in  Numerical  Analysis 
and  Scientific  Computing  Conference,  Kent  State  University,  April  18  21,  2013. 

Invited  Talk. 

•  Iterative  Krylov  Subspace  Methods  for  Sparse  Reconstruction.  International  Workshop  on 
Numerical  Linear  Algebra  with  Applications,  Hong  Kong,  November  14-19,  2013. 

Invited  Talk. 

•  Parallel  Implementation  of  a  Frozen  Flow  Based  Wave-Front  Reconstructor,  Advanced  Maui 
Optical  and  Space  Surveillance  Technologies  (AMOS)  Conference,  Kihei  HI,  2013. 

Poster  Presentation  (given  by  student  co-author  K.  W.  Kelly). 

•  Iterative  Methods  for  Image  Restoration.  4th  Workshop  on  Mathematical  Analysis  on  Inverse 
Problems  and  Imaging,  National  Astronomical  Observatory  of  Japan,  Tokyo,  Japan,  January 
8,  2014. 

Invited  Talk. 

•  Wavefront  Phase  Reconstruction  using  a  Frozen  Flow  Model.  4th  Workshop  on  Mathematical 
Analysis  on  Inverse  Problems  and  Imaging,  National  Astronomical  Observatory  of  Japan, 
Tokyo,  Japan,  January  8,  2014. 

Invited  Talk. 

•  Imaging  Through  Strong  Turbulence.  SIAM  Conference  on  Imaging  Scinece,  Hong  Kong, 
China,  May  12-14,  2014. 

Organized  Minisymposium  jointly  with  S.  M.  Jefferies. 

•  Computational  Methods  for  Inverse  Problems  with  Applications  in  Image  Processing.  Univer¬ 
sity  of  Padova,  Padova,  Italy,  February  17-25,  2014. 

Invited  Short  Course  for  PhD  Students. 

•  Structured  Computations  in  Multi-Frame  Blind  Deconvolution.  Structured  Matrices  and  Ten¬ 
sors:  Analysis,  Algorithms  and  Applications,  National  Taiwan  University,  Taipei,  Taiwan, 
December  8-11,  2014. 

Invited  Talk. 
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•  Tomographic  Reconstruction  of  Atmospheric  Turbulence  from  Microlens  Imagery.  Optical 
Society  of  America  topical  meeting  on  Signal  Recovery  and  Synthesis,  Seattle,  WA,  July  13- 
17,  2014. 

Invited  Talk. 

•  Tomographic  Reconstruction  of  Atmospheric  Turbulence  from  Microlens  Imagery.  SIAM  Con¬ 
ference  on  Imaging  Science,  Hong  Kong,  May  12-14,  2014. 

Invited  Talk. 

•  Fast  Tomographic  Reconstruction  of  Atmospheric  Turbulence  from  Micro-lens  Imagery  Ad¬ 
vanced  Maui  Optical  and  Space  Surveillance  Technologies  (AMOS)  Conference,  September 
9-12,  2014. 

Poster  Presentation. 

•  Numerical  Linear  Algebra  and  Blind  Deconvolution.  Householder  Symposium  XVIV,  Spa, 
Belgium,  June  9-13,  2014. 

Poster  Presentation. 

•  Hybrid  Constrained  Iterative  Methods  for  Inverse  Problems.  SIAM-SEAS  Conference,  Uni¬ 
versity  of  Alabama  at  Birmingham,  March  20-22,  2015. 

Invited  Plenary  Talk. 


Future  research  activities  resulting  from  this  project.  This  project  has  produced  several  im¬ 
portant  research  publications,  and  has  initiated  avenues  of  further  research.  In  particular,  because 
of  this  project,  we  have  been  inspired  to  continue  work  in  the  following  areas: 

•  We  have  an  ongoing  MATLAB  software  project,  called  RestoreTools,  that  uses  an  object 
oriented  approach  to  implement  iterative  image  deblurring  methods.  Graduate  student  Se¬ 
bastian  Berisha  has  recently  worked  on  updating  the  package,  and  has  translated  some  of  these 
codes  into  C-|— I-/MPI  implementations  for  high  performance  computers.  Two  other  students, 
Clarissa  Garvey  and  James  Herring,  are  being  trained  to  make  further  contributions  to  this 
aspect  of  our  work. 

•  The  work  we  have  done  on  using  implicit  filtering  to  avoid  the  local  minimum  trap  in  blind 
deconvolution  problems  has  only  scratched  the  surface  of  what  may  be  possible.  These  meth¬ 
ods  are  expensive  to  implement  for  complicated  blurs,  especially  for  cases  where  the  blur  is 
spatially  variant.  It  is  therefore  important  to  focus  on  spatially  variant  blind  deconvolution 
problems. 

•  An  area  of  great  interest  and  activity  in  the  research  community  is  sparse  reconstructions. 
The  algorithms  for  sparse  reconstructions  are  expensive  for  large  scale  problems,  such  as 
those  that  arise  in  multi-frame  blind  deconvolution.  The  idea  of  using  sparsity  constraints, 
though,  can  be  helpful  in  reducing  some  of  the  problems  associated  with  local  minima,  and 
thus  we  have  been  putting  a  substantial  amount  of  effort  into  this  problem.  We  have  already 
developed  two  different  approaches  for  this  problem;  the  first  one  is  based  on  the  idea  of 
the  Iteratively  Reweighted  Least  Squares  method  and  can  be  efficiently  implemented  in  the 
context  of  Flexible  Krylov  Subspaces;  the  second  one  is  based  on  restarting  the  iterative 
Arnoldi  algorithm.  A  paper  on  this  work  has  already  been  published,  and  we  intend  to 
continue  working  in  this  important  area. 
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•  Related  to  the  previous  item  is  iteratively  reweighted  least  squares  methods  that  account  for 
outliers  in  data.  Our  interest  in  this  area  is  motivated  in  part  by  some  recent  work  by  Dr. 
Brandoch  Calef  (Boeing)  on  iteratively  reweighted  MFBD  methods.  Calef’s  work  shows  great 
promise,  and  we  are  interested  in  developing  this  further,  especially  focusing  on  the  efficient 
computational  aspects. 
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research  is  to  develop  new  computational  approaches  that  help  to  overcome  the  local  minimum  trap  for 
iterative  MFBD  algorithms.  Success  in  this  research  requires  new  approaches  in  several,  interrelated 
areas,  including:  obtaining  a  good  initial  guess,  improved  regularization  methods  and  approaches  for 
choosing  regularization  parameters,  mathematical  analyses  and  efficient  computational  methods  to  reduce 
the  parameter  space,  and  development  of  efficient  implicit  filtering  algorithms  for  large  scale  optimization 
problems. 

In  the  three  years  of  this  grant,  we  have  developed  software,  investigated  the  use  of  implicit  filtering 
methods  to  avoid  local  minima,  considered  Gaussian  Markov  random  fields  to  improve  regularization,  and 
we  have  developed  methods  for  sparsity  constraints  that  can  be  used  for  large  scale  MFBD  problems.  The 
work  done  on  this,  and  a  previous  AFOSR  grant,  initiated  a  very  productive  collaboration  with  Dr.  Stuart 
Jefferies,  IFA,  University  of  Flawaii,  Dr.  Douglas  Flope,  U.S.  Air  Force  Academy,  and  Dr.  Michael  Hart, 
Physics  and  Astronomy,  University  of  Arizona. 

Distribution  Statement 


This  is  biock  12  on  the  SF298  form. 


Distribution  A  -  Approved  for  Public  Release 

Explanation  for  Distribution  Statement 

if  this  is  not  approved  for  public  release,  please  provide  a  short  explanation.  E.g.,  contains  proprietary  information. 
SF298  Form 

Please  attach  your  SF298  form.  A  blank  SF298  can  be  found  here.  Please  do  not  password  protect  or  secure  the  PDF 
The  maximum  file  size  for  an  SF298  is  50MB. 

AFD-070820-035.pdf 

Upload  the  Report  Document.  File  must  be  a  PDF.  Please  do  not  password  protect  or  secure  the  PDF .  The 
maximum  file  size  for  the  Report  Document  is  50MB. 

Report_March_201 5.pdf 

Upload  a  Report  Document,  if  any.  The  maximum  file  size  for  the  Report  Document  is  50MB. 

Archival  Publications  (published)  during  reporting  period: 

A.  Cornelio,  E.  Loll  Piccolomini,  and  J.  G.  Nagy. 

Constrained  Variable  Projection  Method  for  Blind  Deconvolution, 

Journal  of  Physics:  Conference  Series,  volume  386,  issue  1 ,  pages  01 2005,  201 2. 

S.  Berisha  and  J.  G.  Nagy. 

Iterative  Methods  for  Image  Restoration, 

book  chapter  in  Academic  Press  Library  in  Signal  Processing:  Image  Video  Processing  and  Analysis, 
Hardware,  Audio,  Acoustic  and  Speech  Processing, 

Academic  Press,  editor  S.~Theodoridis,  September  2013. 

Q.  Chu,  S.  M.  Jefferies,  and  J.  G.  Nagy. 

Iterative  Wavefront  Reconstruction  for  Astronomical  Imaging, 

SIAM  J.  Sci.  Comput.,  volume  35,  issue  5,  pages  S84-S1 03,  2013. 

J.  M.  Bardsley,  M.  Howard,  and  J.  G.  Nagy. 

Image  Deblurring,  Gaussian  Markov  Random  Fields,  and  Neumann  Boundary  Conditions, 

Elect.  Trans.  Numer.  Anal.,  volume  40,  pages  476-488,  2013. 

S.  M.  Jefferies,  D.  A.  Hope,  M.  Hart  and  J.  G.  Nagy, 

High-resolution  imaging  through  strong  atmospheric  turbulence. 

Proceedings  of  the  SPIE,  volume  8890,  doi:10.1 1 1 7/1 2.20281 93, 

October  25,  2013. 

K.  W.  Kelly  and  J.G.  Nagy. 

Parallel  Implementation  of  a  Frozen  Flow  Based  Wave-Front  Reconstructor, 

AMOS  Technical  Conference,  Kihei,  HI,  2013. 

A.  Cornelio,  E.  Loll  Piccolomini,  and  J.  G.  Nagy. 

Constrained  Numerical  Optimization  Methods  for  Blind  Deconvolution, 

Numerical  Algorithms,  volume  65,  issue  1 ,  pages  23-42,  2014. 

P.  C.  Hansen,  J.  G.  Nagy  and  K.  Tigkos. 

Rotational  Image  Deblurring  with  Sparse  Matrices, 

BIT  Numerical  Mathematics,  54,  pages  649-671 ,2014. 

S.  Gazzola  and  J.  G.  Nagy. 

Generalized  Arnoldi-Tikonov  Method  for  Sparse  Reconstruction, 


SIAM  J.  Sci.  Comput.,  volume  36,  issue  2,  pages  B225-B247,  2014. 


J.  G.  Nagy,  S.  M.  Jefferies,  M.  Hart,  and  D.  A.  Hope. 

Fast  Tomographic  Reconstruction  of  Atmospheric  Turbulence  from  Micro-lens  Imagery, 

Proceedings  of  the  Advanced  Maui  Optical  and  Space  Surveillance  Technologies  (AMOS)  Conference, 
September  9-1 2,  201 4. 

S.  Berisha,  J.  G.  Nagy  and  R.  J.  Plemmons. 

Estimation  of  Atmospheric  PSF  Parameters  for  Hyperspectral  Imaging, 

Numer.  Lin.  Alg.  Appl.,  to  appear,  2015. 

S.  Berisha,  J.  G.  Nagy  and  R.  J.  Plemmons. 

Deblurring  and  Sparse  Unmixing  of  Hyperspectral  Images  using  Multiple  Point  Spread  Functions, 

SIAM  J.  Sci.  Comput.,  to  appear,  201 5. 

Changes  in  research  objectives  (if  any): 

Change  in  AFOSR  Program  Manager,  if  any: 

Extensions  granted  or  miiestones  siipped,  if  any: 

AFOSR  LRiR  Number 
LRiR  Titie 
Reporting  Period 
Laboratory  Task  Manager 
Program  Officer 
Research  Objectives 
Technical  Summary 

Funding  Summary  by  Cost  Category  (by  FY,  $K) 


Starting  FY 

FY-rl 

FY+2 

Salary 

Equipment/Facilities 

Supplies 

Total 

Report  Document 

Report  Document  -  Text  Analysis 

Report  Document  -  Text  Analysis 

Appendix  Documents 

2.  Thank  You 

E-mail  user 

Jul  1 3,  201 5  1 2:01 :30  Success:  Email  Sent  to:  nagy@mathcs.emory.edu 


